BERMIN: A Model Selection Algorithm for Reinforcement Learning Problems

Most reinforcement learning algorithms rely on the use of some function approximation method. In general, their performance will be largely influenced by what function approximation method is used and how it is configured. Current practice is that the user of the algorithm decides about both the method and its configuration, such as the number and nature of basis functions for a linear function approximation method, and tune them in an ad hoc manner. Although good rules of thumb may exist of how to tune a particular method, or which method to use in a particular situation, there is no guarantee that a rule of thumb will give good results on the problem that the user wants to solve. A superior approach, however, is to choose and configure the method based on the data. We show that strong theoretical results can be proven if an appropriate selection procedure is used.1